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Abstract 


Two  of  the  key  activities  in  programs  to  develop  higher  levels  of  helicopter  equipment  reliabil¬ 
ity  are  test  activities:  reliability  problem-identification  testing  and  reliability  demonstration 
testirg.  This  document  (Volume  II  of  a  three-volume  report)  describes  the  variables  that  must 
be  dealt  with  in  reliability  testing:  demonstrated  MTBF*,  desired  level  of  confidence,  demon¬ 
stration  duration,  and  probability  of  passing.  It  also  explains  their  relationships  and  the  testing 
strategy  which  derives  from  the  fact  that  helicopter  hardware  must  have  a  real  MTBF  greater 
than  demonstrated  MTBF  in  order  to  have  a  reasonable  probability  of  passing  a  prescribed 
demonstration.  (Detailed  procedures  and  conclusions  are  contained  in  Volume  I,  Study  Results, 
and  Volume  III,  Sensitivity  Analysis.) 


♦mean  time  between  failures 
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Introduction 


We  have  studied  helicopter  development  testing  to  find  out  what  kinds  of  tests  are  most  effec¬ 
tive  in  improving  the  reliability  of  the  helicopter  dynamic  system  and  how  much  testing  is 
required  to  meet  various  reliability  levels. 

Two  kinds  of  development  tests  are  the  key  to  the  achievement  of  component  reliability: 
problem-identification  tests  and  demonstration  tests. 

•  Problem-Identification  Tests  are  designed  to  find  problems  so  that  they  can  be  fixed. 

•  Demonstration  Tests  are  conducted  after  the  problem-identification  tests  have  been 
completed,  primarily  to  evaluate  the  resultant  reliability  level  and  to  formulate  a  state¬ 
ment  of  what  it  is. 

All  of  the  development  tests  are  described  in  Volume  I.  This  volume  (Volume  II)  concentrates 
on  the  relationships  between  the  problem-identification  tests  and  the  demonstration  tests.  The 
reader  is  encouraged  to  review  the  detailed  procedures  and  conclusions  included  in  Volume  I, 
Study  Results  and  Volume  III,  Sensitivity  Analysis. 

Figure  1  shows  how  reliability  grows  by  a  process  of  analysis  and  testing.  This  process  is  de¬ 
signed  to  demonstrate  that  contractual  reliability  requirements  have  been  met.  Before  the 
development  program  begins,  the  user  of  the  equipment  and  the  manufacturer  of  the  equipment 
numerically  analyze  the  process  (going  in  reverse  order  from  demonstrated  MTBF)  in  order  to 
estimate  the  magnitude  of  each  element.  Our  discussion  here  will  also  take  the  steps  of  the 
process  in  reverse  order. 
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Reliability  Demonstration  Testing  \ 


A  reliability  demonstration  is  a  test  to  show  that  hardware  meets  reliability  requirements.  A 
small  quantity  of  specimens  (called  a  “sample”)  is  tested  for  a  limited  amount  of  time,  and  the 
results  are  evaluated  by  the  application  of  probability  theory  to  establish  the  reliability  of  the 
total  quantity  of  jiprts  (called  “the  population”).  , 

Demonstration  testing  is  performed  near  the  end  of  the  development  cycle  to  ascertain  that  the 
product  meets  the  reliability  requirement.  Reliability  demonstration  testing  is  performed,  not 
to  find  problems,  but  to  show  that  the  problems  have  been  eliminated,  or  that  only  an  accept¬ 
able  number  of  problems  remain.  The  testing  must  be  conducted  in  'such  a  way  that: 

•  There  is  no  doubt  about  the  environment  being  encountered.  • 

•  Hardware  configuration  is  known  and  kept  constant. 

•  What  constitutes  failure  is  unambiguously  defined. 

•  All  events  are  recorded  accurately. 

a 

There  are  two  types  of  reliability  demonstration:  hypothesis  testing  and  interval-estimation 
testing.  The  major  difference  between  the  two  types  lies  in  the  evaluation  of  the  test  results; 
the  testing  of  hardware  is  the  same  for  both. 

HYPOTHESIS  TESTING 

Hypothesis  Resting  is  the  type  of  testing  described  by  MIL-STD-78 1 .  It  uses  the  number  of 
failures  and  the  operating  time  to  test  the  hypothesis  that:  “This  is  a  good  lot”.  Goodness  is 
defined  as  having  a  demonstrated  MTBF  at  least  as  high  as  the  MTBF  goal.  Levels  of  decision 
risk  are  identified  by  statistical  process.  The  risk  that  threatens  both  the  user  and  manufacturer 
is  that  of  erroneously  accepting  bad  hardware  and  rejecting  good  hardware. 

INTERVAL-ESTIMATION  TESTING 

Interval-estimation  uses  the  number  of  failures  and  the  operating  time  to  categorize  hardware 
reliability  as  some  specific  numerical  interval.  At  the  same  time,  it  assigns  to  the  estimate  a 
probability  of  being  correct,  called  a  confidence  level.  \ 

V 

Interval-estimation  testing  does  nbt  usually  provide  ak  early  an  accept/reject  decision  as  does 
hypothesis  testing.  For  this  reason,  higher  test  costs  may  be  involved  where  interval-estimation 
testing  is  used.  This  disadvantage  is  offset  by  the  ability  of  interval-estimation  tests  to  produce 
numerical  values,  which  are  of  value  to  logistics  planners. 

1  '  '  \  i 

Volume  I  of  this  report  dealfc  exclusively  with  interval-estimation  testing.  For  this  reason,  the 
remainder  of  Volume  II  considers  interval-estimation  testing  exclusively. 


\ 
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\  -  ; 

For  example,  having  observed  four  specimen  failures  in  8000  hours  operating  time,  we  can  state 
with  90-percent  confidence  (i.e.,  nine  times  out  of  ten  1  will  be  correct)  that  the  demonstrated 
MTBF  of  the  product  is  1000  hours  or  greater:  it  lies  between  1000  hours  and  infinity.  And 
that  is  not  the  only  statement  we  can  make.  Statistical  techniques  produce  not  one  but  a  family 
of  alternative  statements  from  the  same  test.  Also,  at  any  confidence  desired,  demonstrated 
MTfiF  can  be  expressed  with  either  a  finite  or  an  infinite  upper  limit: 


Demonstrated  MTBF  (hours) 


Confidence 

Finite  Lower  Limit, 

Finite  Upper  and 

Level 

Infinite  Upper  Limit 

Lower  Limits 

99% 

689  or  greater 

635  to  7422 

95% 

874  or  greater 

781  to  4928 

90% 

1001  or  greater 

874  to  4061 

85% 

1101  or  greater 

942  to  3599 

80% 

1 190  or  greater 

1001  to  3289 

75% 

1 275  or  greater 

1053  to  3057 

74% 

1292  or  greater 

1062  to  3016 

73% 

1 308  or  greater 

1072  to  2979 

70% 

1358  or  greater  \ 

1101  to  2872 

60% 

1528  or  greater 

11 90  to  2589 

50% 

1712  or  greater 

1275  to  2375 

35% 

^  2056  or  greater 

1400  to  2166 

1 

If  the  same  test  results  can  produce  a  great  many  alternative  demonstrated  MTBF  intervals,  it  is 
also  true  that  any  one  demonstrated  MTBF  interval  could  have  been  produced  by  a  great  many 
alternative  combinations  of  test  results.  For  example,  the  resultant  90-percent  confidence  that 
the  demonstrated  MTBF  is  1 000  hours  or  more  could  have  been  produced  by  any  one  of  the 
following  sets  of  test  results: 


\ 
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\ 

\ 

\ 

\ 


0  failures  in  2302  hours 

1  failure  in  3890  hours 

2  failures  in  5322  hours 

3  failures  in  668 1  hours 

4  failures  in  7996  hours  ( 'v  8000) 

5  failures  in  9275  hours 


HOW  MANY  FAILURES  CAN  WE  HAVE? 

Figure  2  plots  confidence  level  against  the  amount  of  time  on  test  (demonstration  duration)  for 
a  demonstrated  MTBF  of  at  least  1000  hours.  Consider  the  first,  or  zero-failure,  line.  It  says 
that  if  we  test  for  500  hours  without  a  failure,  we  are  40-percent  sure  that  demonstrated  MTBF 
will  be  at  least  1000  hours.  If  we  test  for  2302  hours  without  a  failure,  we  just  reach  90-percent 
confidence.  However,  suppose  that  by  the  time  we  reach  2302  hours  of  testing,  we  have  one 
failure.  The  confidence  level  drops  to  67  percent.  To  reach  90-percent  confidence  with  one 
failure,  we  will  need  3890  hours  of  testing. 

100 

•0 


70 

•0 

CONFIDENCE 

LEVEL. 

%  (THAT  MTBF  90 
g  1000  HOURS) 

40 

JO 

30 

10 

0 

0  2  4  0  0  10  12  14  10  It 

DEMONSTRATION  DURATION  (1000  HOURS) 


Figure  2.  Determining  Allowable  Number  of  Failures  To  Demonstrate  an  MTBF  >1000  Hours 
at  a  Confidence  Level  >90%. 

Note  that  if  we  test  for  3890  hours  with  zero  failures,  we  will  have  actually  demonstrated  that 
MTBF  is  equal  to  or  greater  than  1000  hours  at  98-percent  confidence  level.  Thus,  regardless  of 
whether  we  have  zero  or  one  failure  in  3890  hours  of  testing,  we  are  fully  justified  in  stating 
that  an  MTBF  equal  to  or  greater  than  1 000  hours  has  been  demonstrated  at  a  confidence  level 
of  at  least  90  percent. 

Suppose  we  choose  a  test  duration  of  6000  hours.  Three  failures  will  give  us  a  confidence  level 
of  85  percent.  For  anything  more  than  three  failures,  the  confidence  level  is  still  lower.  The 
largest  number  of  failures  which  allows  us  a  confidence  equal  to  or  greater  than  90  percent  is 
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two  failures:  two  failures  corresponds  to  94-percent  confidence.  One  failure  gives  98-percent 
confidence,  and  zero  failures  gives  99.75-percent  confidence.  Thus,  two  failures  c  less  in  6000 
hours  of  testing  guarantees  at  least  90-percent  confidence  that  the  demonstrated  MTBF  is  at 
least  1000  hours. 

It  is  this  changing  number  of  allowable  failures  that  accounts  for  the  step  or  saw-tooth  patte  n 
of  Figure  2  and  subsequent  figures. 

A  WORD  ABOUT  TERMINOLOGY 

MTBF  values  developed  by  testing  are  commonly  called  demonstrated  MTBFs.  Actually,  we  do 
not  demonstrate  MTBF;  rather,  we  demonstrate  that  the  MTBF  falls  within  certain  ranges,  such 
as  “between  1500  and  10,000  hours”  or  “greater  than  2500  hours”,  with  some  numerical  level 
of  confidence  that  this  is  true.  Since  it  is  cumbersome  to  continually  refer  to  the  “range  within 
which  the  hardware’s  MTBF  has  been  demonstrated  to  fall,”  we  use  the  accepted,  convenient, 
but  nondescriptive  term  “demonstrated  MTBF.”  However,  we  should  remember  the  meaning 
behind  the  term. 
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Plans  for  Testing 


When  a  hardware  development  contract  calls  for  “reliability  demonstration  at  a  confidence 
level  of  Y-percent  that  the  MTBF  of  the  hardware  is  equal  to  or  greater  than  X  hours”,  what  do 
we  do?  What  factors  must  we  consider  to  ensure  that  the  demonstration  requirement  will  be 
met? 

In  order  to  plan  the  demonstration  tests  and  problem-identification  tests,  we  must  determine 
what  real  MTBF  is  necessary  and  how  it  can  be  achieved.  Answering  this  question  involves  an 
examination  of  the  four  variables  which  determine  the  real  MTBF: 

•  The  MTBF  to  be  demonstrated 

•  The  confidence  level  at  which  it  is  to  be  demonstrated 

•  The  duration  of  the  demonstration  test 

•  The  probability  of  passing  the  demonstration  test 


HARDWARE 


DEMONSTRATION  TEST 


CONCLUSIONS 


REAL  MTBF 
IFOR  THE  SAMPLE) 


COUNTS: 

'  SAMPLE  FAILURES 
•  OPERATING  HOURS 


YIELDS: 

■  DEMONSTRATED  MTBF 
•  CONFIDENCE  LEVEL 
IFOR  THE  POPULATION) 


Figure  3.  Demonstration  Process  Relationships. 


Figure  3  depicts  the  relationships  in  the  demonstration  process.  We  will  now  examine  each  of 
the  four  variables. 

THE  MTBF  TO  BE  DEMONSTRATED 

The  value  of  demonstrated  MTBF  selected  by  the  user  of  the  equipment  establishes  the  neces¬ 
sary  value  of  real  MTBF  in  accordance  with  the  typical  relationship  shown  in  Figure  4.  The 
higher  the  demonstrated  MTBF,  the  higher  the  real  MTBF  entering  the  test  must  be. 


RIAL  MTV 
(HOURS) 


OCMOMCT  RATIO  MTV  (HOURS) 


Figure  4. 

Real  MTBF  Must  Be  Increased  To  Pro¬ 
duce  Higher  Demonstrated  MTBF 
(When  OtLjr  Test  Variables  Remain 
Constant). 


Figure  5. 

Value  of  the  Real  MTBF  Must  Be  In¬ 
creased  for  Greater  Confidence  Level 
(When  Other  Test  Variables  Remain 
Constant). 
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CONFIDENCE  LEVEL 

How  sure  are  you?  90-percent?  95-percent?  Confidence  level  expresses  the  probability  of  our 
being  correct  when  we  make  a  statement  about  a  hardware  characteristic  such  as  MTBF.  Cer¬ 
tainty  is  100  percent.  The  confidence  level  that  we  have  in  a  demonstrated  MTBF  calculated 
from  test  results  will  always  be  less  than  a  certainty.  The  confidence  level  selected  by  the  user 
establishes  the  necessary  value  of  real  MTBF  in  accordance  with  the  relationship  shown  in 
Figure  5.  The  higher  the  confidence  level  desired,  the  higher  the  real  MTBF  must  be. 

DURATION  OF  THE  DEMONSTRATION  TEST 

Demonstration  duration  is  the  sum  of  all  of  the  test  hours  on  all  the  hardware  specimens  in  the 
demonstration.  After  this  number  of  hours  has  been  accumulated,  we  count  failures  and  draw 
conclusions.  The  demonstration  duration  agreed  upon  by  the  usei  and  the  manufacturer  estab¬ 
lishes  the  necessary  value  of  real  MTBF  in  accordance  with  the  relationship  shown  in  Figure  6. 
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DEMONSTRATION  DURATION  (1000  HOURS) 


Figure  6.  Necessary  Value  of  the  Real  MTBF  Decreases  as  Demonstration  Duration  Increases 
(When  Other  Test  Variables  Remain  Constant) . 


This  relationship  is  usually  more  difficult  to  understand  than  the  first  two;  it  reflects  the  way 
the  statistics  try  to  keep  luck  from  affecting  the  results.  Short  tests  must  have  fewer  failures 
per  hour  than  longer  tests  to  demonstrate  the  same  MTBF.  The  longer  the  test,  the  lower  the 
real  MTBF  need  be. 

PROBABIL ITY  OF  PASSING 

Regardless  of  how  high  the  real  MTBF  may  be,  there  is  always  less  than  a  certainty  (less  than 
100-percent  probability^  that  the  component  will  pass  any  given  demonstration.  For  one  thing, 
demonstration  tests  are  not  perfect.  There  is  some  probability  that  a  test  will  reject  go  cl  hard¬ 
ware  and  pass  bad  hardware  because  the  behavior  of  the  sample  may  not  be  typical  of  the 
behavior  of  the  population. 

The  probability  of  passing  a  demonstration  test  can  be  calculated.  It  increases  as  the  real  MTBF 
is  increased;  it  establishes  the  necessary  value  of  real  MTBF  in  accordance  with  the  relationship 
shown  in  Figure  7. 
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Figure  7.  Probability-of-Passing  Demonstration  Increases  With  Value  of  Real  MTBF  (When 
Other  Test  Variables  Remain  Constant). 
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Both  the  user  and  the  manufacturer  want  high  probability-of-passing  values:  the  user  because 
he  wants  to  put  his  new  helicopter  into  operation  on  schedule;  the  manufacturer  because  he 
wants  to  get  paid  on  schedule.  The  user  also  realizes  that  high  probability-of-passing  values  re¬ 
quire  high  real  MTBF,  which  will  ultimately  reduce  his  costs  of  operation  and  maintenance. 
However,  the  higher  real  MTBFs  are  achWed  only  through  more  development  effort,  which 
will  require  more  money.  One  must  compare  costs  and  benefits  of  the  higher  MTBFs  in  order 
to  determine  how  high  the  probability  of  passing  should  be. 

HOW  MANY  TEST  SPECIMENS? 

We  would  like  to  have  enough  test  specimens  to  accommodate  all  reasonable  failure-frequency 
distributions:  infant  mortelity.  random  failure,  and  wearout.  This  implies  both  short  test  times 
oh  a  large  number  of  specimens  (infant  mortality)  and  high  times  on  just  a  few  specimens 
(wearout).  These  are  conflicting  requirements. 

However,  schedule  restraints  will  usually  resolve  the  conflict.  On  the  typical  three-  to  six-year 
development  program,  high  utilization  of  test  stands  is  essential.  During  the  initial  testing,  when 
infant  mortality  modes  are  being  encountered,  a  great  number  of  test  specimens  is  needed.  As 
problems  are  identified  and  corrected,  MTBF  is  improved,  and  specimens  run  longer  between 
failures.  A  few  components  then  tend  to  accumulate  a  large  number  of  test  hours  during  the 
final  portions  of  the  program  and  the  wearout  modes  are  identified. 

For  demonstration  testing,  the  number  of  test  specimens  is  less  critical.  Most  of  the  problems 
have  been  fixed  by  then,  and  the  few  remaining  failure  modes  are  likely  to  be  exponentially 
distributed.  The  failure  rate  is  constant  with  hardware  age.  When  this  distribution  is  present, 
the  number  of  test  specimens  does  not  affect  test  results  so  long  as  the  specimens  are  reason¬ 
ably  representative  of  the  range  of  allowable  production  variation.  Five  units,  each  running 
for  100  hours,  will  produce  the  same  number  of  failures  as  one  unit  running  500  hours.  His¬ 
torical  data  indicate  that  most  major  dynamic  components  have  close  to  an  exponential  dis¬ 
tribution,  so  that  the  number  of  test  specimens  is  not  critically  important  to  a  demonstration 
test.  The  demonstration  programs  suggested  in  Volume  I  employ  1 5  to  30  different  specimens  - 
which  is  adequate  for  evaluation  purposes. 

NECESSARY  VERSUS  ACHIEVED  VALUES  OF  REAL  MTBF 

The  previous  discussions  dealt  with  identification  of  the  necessary  value  of  real  MTBF.  In  so 
doing,  we  may  have  given  the  impression  that  we  know  or  can  readily  measure  the  achieved 
value  of  real  MTBF.  We  can't.  To  measure  the  achieved  value,  we  would  have  to  run  every  com¬ 
ponent  to  failure,  never  repair  any  failed  units,  and  never  build  any  additional  units. 

Some  say  that  we  should  be  able  to  calculate  the  achieved  value  of  real  MTBF  from  test  failure 
count,  operating  hours,  and  a  known  probability  of  passing.  Unfortunately,  demonstration  tests 
do  not  yield  single  statements  of  the  achieved  value  of  real  MTBF  and  probability  of  passing. 
They  involve  whole  families  of  statements,  as  we  have  seen.  Therefore,  since  one  specific  proba¬ 
bility  of  passing  is  never  known,  the  discrete  real  MTBF  present  cannot  be  established  The  best 
we  can  do  is  estimate  the  magnitude  of  the  achieved  value  through  the  use  of  historical  data  on 
similar  hardware,  design  analysis,  review  of  test  results,  and  engineering  judgment. 
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Hardware  Development 


After  one  has  identified  the  necessary  value  of  real  MTBF,  the  next  problem  is  “How  do  you 
get  hardware  up  to  that  level?” 

Two  activities  contribute  to  improving  hardware  reliability  during  the  development  phase.  The 
first  is  the  analytical  process  (which  includes  certain  specialized  tests)  perforn  ed  in  support  of 
design.  This  process  includes  determining  sizes,  selecting  design  allowables,  making  detailed 
drawings,  reviewing  the  design,  and  predicting  reliability  levels.  The  second  involves  problem- 
identification  testing  of  the  hardware  to  detect  failure  modes.  Both  of  these  activities  identify 
deficiencies  that  would  cause  unreliability. 

It  is  acknowledged  that  problem-identification  testing  overlaps  the  analytical  process.  Both  ac¬ 
tivities  have  the  ability  to  prevent  problems  from  reaching  the  field,  and  most  development 
programs  employ  the  two  activities  in  combination.  However,  we  apply  testing  as  a  backup 
process  because  today’s  analytical  design  support  process  cannot  completely  eliminate  reliabil¬ 
ity  problems.  The  analytical  method  holds  long-range  potential  for  producing  high  levels  of 
reliability  more  cost-effectively,  but  with  the  technology  available  today,  we  have  to  rely  on 
testing  to  meet  the  reliability  goals. 

DESIGN  AND  ANALYSIS 

Design  and  analysis  activities  are  those  activities  performed  during  the  design  phase  (prior  to 
the  building  and  testing  of  hardware)  to  improve  the  reliability  of  the  hardware.  They  include: 

•  Development  of  criteria 

•  Apportionment  of  goals 

•  Development  of  specifications 

•  Prediction  of  reliability 

•  Analysis  of  failure  modes  and  their  effects 

•  Formal  design  review 

Also  included  in  this  activity  is  a  series  of  specialized  tests  called  design  development  tests  that 
occur  early  in  the  design  phase. 

DESIGN  DEVELOPMENT  TESTING 

Not  all  testing  directly  improves  reliability.  Volume  I  used  the  term  design  development  testing 
to  describe  testing  that  contributed  at  best  indirectly  to  improved  reliability.  Design  develop- 
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ment  testing  includes  material  testing  (sometimes  under  extreme  environments),  fatigue  testing 
(to  define  the  shape  of  the  stress- versus-cycles-to-failure  curve),  and  performance  testing. 


MTBF  OFF-THE-BOARD 

The  MTBF  off-the-board  is  the  reliability  prediction  produced  by  the  design  and  analysis  activ¬ 
ities  before  any  problem-identification  testing.  Reliability  is  predicted  from  historical  data  on 
similar  hardware  and  analysis  of  the  design  to  identify  suspected  failure  modes  and  their  fre¬ 
quencies.  The  prediction  process  requires  some  application  of  engineering  judgment. 

If  the  design  and  analysis  activity  were  completely  effective,  we  would  develop  an  MTBF  off- 
the-board  that  would  equal  the  real  MTBF  and  hence  satisfy  the  requirements  for  entering 
demonstration  testing.  Since  design  analysis  is  not  yet  that  effective,  we  achieve  the  necessary 
value  through  testing  to  identify  problems,  and  then  take  the  necessary  corrective  action. 

PROBLEM-IDENTIFICATION  TESTING 

Problem-identification  testing  is  endurance  testing  performed  to  identify  reliability  problems 
and  subsequently  verify  the  effectiveness  of  corrective  action.  It  attempts  to  duplicate  the 
equipment’s  duty  cycle  and  operating  environment  for  long  periods  of  operation.  We  use  his¬ 
torical  relationships  between  test  duration  and  MTBF  improvement  in  conjunction  with  the 
MTBF  off-the-board  to  determine  how  long  the  problem-identification  test  must  be  in  order 
to  achieve  the  real  MTBF. 

Figure  8  shows  that  real  MTBF  increases  as  a  result  of  problem-identification  testing  and  cor¬ 
rective  action.  Determining  this  relationship  for  a  specific  component  being  subjected  to  a 
specific  test  technique  is  a  complex  process.  For  the  design  coming  off  the  drawing  board,  it 


Figure  8.  Real  MTBF  Increases  as  a  Result  of  Problem  Identification  Testing  and  Corrective 

Action  (Example  Shown:  Single-Rotor  Helicopter  Main  Transmission  in  “Iron-Bird” 
Dynamic  System  Test  Rig). 
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requires  analysis  of  (1)  failure  modes  and  frequencies,  (2)  the  effectiveness  of  specific  test  tech¬ 
niques  in  identifying  specific  problems,  and  (3)  the  degree  to  which  each  observed  problem  can 
be  eliminated  by  a  design  change.  In  the  basic  study  (Volume  1),  discrete  values  were  used  to 
evaluate  each  of  these  factors.  The  effect  of  choosing  different  values  for  these  factors  is  ex¬ 
plored  in  the  Sensitivity  Analysis  in  Volume  III.  Factors  explored  in  Volume  III  include: 

•  MTBF  off-the-board 

•  The  effectiveness  of  individual  test  techniques  in  detecting  problems 

•  The  number  of  times  a  problem  must  be  observed  in  problem-identification  testing  to 
postulate  that  it  has  been  corrected. 


Review 


This  document  explains  some  of  the  statistical  theory  used  to  evaluate  reliability  demonstration 
testing.  The  results  of  the  interval-estimation  type  of  reliability  demonstration  testing  arc  a 
range  and  a  confidence  level. 

In  order  to  have  a  reasonable  probability  of  passing  a  prescribed  demonstration  within  reason¬ 
able  test  durations,  hardware  needs  to  be  developed  with  a  real  MTBF  greater  than  the  demon¬ 
strated  MTBF.  Determining  the  necessary  value  of  real  MTBF  requires  consideration  of  the 
probability  of  passing,  the  demonstration  duration,  the  desired  level  of  confidence,  and  the 
demonstrated  MTBF. 


The  process  for  developing  the  necessary  value  of  real  MTBF  involves  design  and  analysis  activ¬ 
ity  and  reliability  problem-identification  testing.  A  display  of  these  relationships  is  shown  in 
Figure  9,  an  expansion  of  the  schematic  with  which  we  began  this  volume. 
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Figure  9.  Hardware  Reliability  Growth  Process. 
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With  this  background,  the  reader  may  wish  to  review  Volume  1,  Study  Results,  and  Volume  III, 
Sensitivity  Analysis.  These  documents  tell  how  much  reliability  problem-identification  testing 
is  appropriate  for  various  demonstration  requirements;  they  also  give  typical  test-program  costs. 
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Q-lossary 


MTBF 

The  term  Mean  Time  Between  Failures  (MTBF)  pertains  to  hardware  failure.  Failure  usually 
describes  a  condition  where  the  hardware  no  longer  functions  properly,  and  consequently  re¬ 
quires  some  form  of  unscheduled  maintenance  to  restore  the  item  (or  the  aircraft)  to  the 
properly  functioning  condition. 

i 

MTBUR 

Sometimes  thy  necessary  unscheduled  maintenance  can  be  accomplished  without  removihg  the 
failed  item  from  the  aircraft.  For  major  dynamic  components,  the  failed  item  is  usually  re- 
moyed  from  the  aircraft,  repaired,  and  reinstalled;  or  the  aircraft  itself  is  restored  to  a  properly 
functioning  condition  by  replacing  the  failed  item  with  a  properly  functioning  one.  The  term 
Mean  Time  Between  Unscheduled  Removals  (MTBUR)  pertains  to  removals  of  failed  hardware 
from  the  aircraft.  ,  j 

MTBR 

It  has  been  common  practice  to  remove  properly  functioning  items  from  the  aircraft  on  a 
scheduled  basis  for  teardown,  inspection,  and  overhaul,  to  prevent  or  reduce  the  frequency  of 
subsequent  failures  during  flight.  The  term  Mean  Time  Between  Removals  (MTBR)  considers 
both  the  unscheduled  removals  (due  to  failure)  and  the  scheduled  removals  (for  preventive 
maintenance).  \ 

TBO 

The  scheduled  removal  of  items  for  preventive  maintenance  (teardown,  inspection,  and  over¬ 
haul)  occurs  at  a  time  referred  to  as  the  TBO  Interval,  where  TBO  repfesents  Time  Between 
(scheduled)  Overhauls. 

I 

NOTE:  Usage  of  MTBR  and  MTBF 

Volume  I  dealt  exclusively  with  major  dynamic  components  installed  in  helicopters. 
This  relatively  expensive,  complex  hardware  neaily  always  requires  removal  from  the 
aircraft  for  repair  or  replacement  in  order  to  restore  the  aircraft  to  a  properly  func¬ 
tioning  condition.  Further,  scheduled  removal  for  preventive  maintenance  is  still 
practiced,  although  the  goal  for  future  designs  is  to  eliminate  or  reduce  this  practice. 
The  term  MTBR  was,  therefore,  appropriate  to  use  in  Volume  I. 
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In  preparing  Volume  II,  it  was  desired  to  present  maximum  applicability  to  hardware 
demonstration  in  general,  (i.e.,  both  “items”  and  “items  installed  in  aircraft”  can 
properly  be  subjected  to  reliability  demonstration).  For  this  reason,  the  term  MTBF 
was  deemed  more  appropriate  for  Volume  11. 


MTBF  OFF-THE-BOARD 

The  reliability  prediction  produced  by  the  design  and  analysis  activities  before  problem- 
identification  testing.  (See  page  13.) 

i 

REAL  MTBF 

The  actual  reliability  of  the  component.  Particular  usage  of  the  term  will  depend  on  whether 
reference  is  being  made  to  necessary  or  achieved  values.  (See  page  1 1 .) 


DEMONSTRATED  MTBF 

A  range  of  values  developed  by  testing  within  which  the  MTBF  falls.  The  MTBF-to-be-demon- 
strated  is  a  range  selected  by  the  user  before  demonstration  testing  begins.  (See  page  6.) 


PROBLEM  IDENTIFICATION  TESTING 

Testing  to  find  problems-so  that  they  can  be  corrected.  (See  page  13.) 

i  .  I 


DEMONSTRATION  TESTING 

Testing  after  problem  identification  apd  correction  to  evaluate  resultant  reliability.  (See 
page  3.)  !  . 

\  | 

CONFIDENCE  LEVEL  1 


Probability  that  a  specific  range  of  numerical  values  estimated  for  demonstrated  MTBF  will  be 
correct.  (See  page  9.) 

\  '  ' 

PROBABILITY  OF  PASSING 

Degree  of  certainty  that  the  component  will  pass  a  given  demonstration.  (See  page  10.) 

! 

DEMONSTRATION  DURATION 

The  sum  of  all  <j>f  the  test  hours  for  all  of  the  hardware  specimens  in  the  demonstration.  (See 
page  9.) 


